When compressed video is sent from a source to a destination over a network, lost packets can cause visible errors. At the source, a source coder typically uses temporal predictive coding, also called interframe coding wherein a difference between a frame and a reference picture is coded. Such an interframe coded frame is sometimes called an INTER frame. In the case of packet communication to a destination point, if there is an error, e.g., a missed packet, errors can propagate to subsequent frames. One solution is for the source to occasionally but sufficiently frequently send an independently coded frame, e.g., that is not dependent on some past frame, e.g., an intraframe coded frame, also called an INTRA frame. An independently coded frame is also called an I-frame. I-frames, however, use more bits than INTER frames, so to increase compression, it is desirable to not send I-frames.
When communication from the destination back to the source of communication is possible, error resilient methods are known that use feedback from a decoder as to whether or not there has been an error such as a lost packet. An encoder receiving such feedback can correct the error, e.g., by encoding the next frame in such a way that it does not reference the erroneous data. One method is for the encoder to send an I-frame upon being informed of an error. A more efficient method is for the encoder to start using a different, e.g., earlier frame as the reference frame for temporal predictive coding. In such a case, the encoder needs to indicate to the decoder that it has done so, and which frame is the reference frame for predictive coding such that the encoder and decoder can maintain synchronization about which frame is being used as a reference picture for interframe coding. This can help avoid using intra-coded I-frames.
Known methods for using feedback to provide reference image error resilience deal with a single transmitting encoder and single receiving decoder. With the advent of high definition, decoding systems are known that use many decoders, e.g., many programmable processors (“decoding processors”) that each decode different portions of the image simultaneously. The encoder and the decoders each include a memory that maintains a data structure for storing reference images used for coding/decoding interframe coded data. Each decoder can provide feedback to indicate to the encoder whether or not it has received data. At the source, the encoder can respond by sending instructions to the decoders at the destination to use one or another reference picture. A problem occurs when some but not all decoders correctly receive such instructions. The encoder needs to be sure that all decoders at the destination are using the same appropriate reference picture.
A similar problem arises in a multipoint video teleconferencing system. Suppose an encoder at a source transmits coded video to a plurality of destinations that each has at least one decoder. The source encoder and each destination's decoder or decoders each includes a memory with a data structure for storing reference images used for coding/decoding interframe coded data. Each destination decoder can provide feedback to indicate to the source encoder whether or not the destination has correctly received data. The source encoder can respond by sending instructions to use one or another reference picture at each destination's decoder. A problem occurs when some but not all destinations, e.g., each decoder at each destination, correctly receive such reference picture instructions. The source encoder in such a situation needs to be sure that all destination decoders are using the same appropriate reference picture.